ftp.cs.arizona.edu/ftp.cs.arizona.edu.tar/ftp.cs.arizona.edu/icon/newsgrp/group98b.txt/000103_icon-group-sender _Wed Jun 24 08:08:45 1998.msg

Internet Message Format | 2000-09-20 | 11KB

Return-Path: <icon-group-sender>
Received: from kingfisher.CS.Arizona.EDU (kingfisher.CS.Arizona.EDU [192.12.69.239])
    by baskerville.CS.Arizona.EDU (8.8.8/8.8.7) with SMTP id IAA01699
    for <[e-mail]>; Wed, 24 Jun 1998 08:08:44 -0700 (MST)
Received: by kingfisher.CS.Arizona.EDU (5.65v4.0/1.1.8.2/08Nov94-0446PM)
    id AA29429; Wed, 24 Jun 1998 08:08:34 -0700
From: [e-mail] (Frank Sergeant)
To: [e-mail]
Subject: Re: using Icon for database application
Date: Tue, 23 Jun 1998 16:33:31 -0500
Reply-To: [e-mail]
Message-Id: <[e-mail]>
In-Reply-To: <[e-mail]>
Lines: 233
Errors-To: [e-mail]
Status: RO
Content-Length: 10427

Gordon, I very much appreciate your comments and suggestions.
I don't think I can agree completely yet, but you've given
me food for thought.

[e-mail] (Gordon Peterson) wrote:
> > xBASE language) and consists of around 150,000 lines of
> > source.  It runs under DOS (or a DOS box in Windows).
>
> That's a fairly huge Clipper application... I'd think that if it
> were written really utilizing the tools present to best advantage
...
> that it would be able to be a whole lot smaller than that.

     I agree that it is large and that it could/should be
smaller.  I attribute this to my predecessors on the project.
There has been a good bit of redundancy in the system.  For
example, when I took it over I found 6 monthly statement
routines: 1 set of 3 for each of 3 possible forms for
printing a single statement and a similar set of 3 routines
for printing all statements.  Obviously, this multiplies the
opportunities for errors and increases the difficulty of
wrapping one's mind around the system.  All reports were
written individually, duplicating the line counting and
report heading logic each time.  Etc.  I have gradually been
factoring out the commonalities but a lot of work remains.
I understand how such (unfactored) code comes about: there
is no money to be made yet there is an obligation to do
something (or at least appear to do something) to solve a
current emergency, and so the most hurried quickest immediate
fix is applied without regard to the long range maintenance
problems it creates, cutting and pasting rather than
analyzing and simplifying.


> >... (Any workstation crashing puts the data files on the server at
> risk.)
>
> I've written a program (in S*BOL) which reads through xBASE
> .DBF files and "repairs" this kind of "not a database file"
> problems.  It fixes many kinds of scrogged fields in the
> header, and particularly fixes the length irregularities
> which usually create this symptom.

     I'm going to quote two more parts of your message out of
order so they will be near the above quote.

> reliable.  In all the time I've been working with FoxPro
> (for DOS) I have seen few (if any) legitimate BUGS in the
> FoxPro package itself.  It seems to overall be very stable.

> I think that, from a reliability and support standpoint,
> it's in most cases just plain dumb to go to a mixed-OS
> environment.

     I'm not sure it is a bug in the FoxPro package (or in
Clipper), but _how come you have those S*BOL repair
programs_ if FoxPro and the environment (and the OS) are so
damn stable?  I think there are several factors.  One is
whether the hardware, especially the RAM, is fully reliable
or partially (intermittently) defective.  The best method I
know of to determine this is to run what I call the Linux
stress test (compile the Linux kernel).  I've written up a
little article about this ("Stress Testing Hardware") and
put it on my web site.  So, being able to run (and later
rerun) this stress test to verify the hardware is a
beneficial side effect of having Linux available.  Also, I
am convinced Linux is a more reliable operating system than
W95 (and perhaps WNT).

     Another factor in data integrity is what I was
referring to when I called it "putting all my eggs in
one basket", but that isn't really the best analogy.
The eggs were already in one basket, that is, the
application's data resided on a single machine, the
file server.  The problem is that all the workstations
independently access that data.  If any one of those
machines fails, the data is at risk.  Putting a UPS
on the server isn't enough, etc.  But, if I move to
"client/server" then it seems I could arrange things
so nothing the clients do can endanger the data on
the server.  Therefore, I remove several potential
points of failure.

     Another factor _may_ be bugs in W95.  It was
interesting to read over the bug history list for
Kermit.  Item after item referred to bugs they had
located in W95 that were not present either in OS/2
or in WNT.  With regard to file server record locking
and such, there is the W95 "vredir" problem and several
"fixes" from Microsoft and several Knowledge Base articles
about it.  The Knowledge Base articles are gibberish as
far as I am concerned.

     So, it may be dumb to go to a mixed OS environment.
But, it may be dumber to use Windows by itself.  Anyway,
I'm not saying you are wrong about this.  I've had my own
strong reservations about recommending Linux for my
customers due to the support problem.  I have figured that
if W95 screws up, they might perhaps blame Microsoft, but
if Linux screws up they will blame me.  Nevertheless, I
have talked customers through (or have gone to the office
to do it myself) various reconfigurations of W95 and its
networking.  I am beginning to think Linux cannot cause
me any more trouble than W95 is causing me.  Plus, Linux
should be much easier to administer remotely, and that
could be worth a lot.  So, I am not saying I will demand
customers use a Linux server, just that I see a possible
advantage to keeping my options open.  (Although it was
W3.11, not W95, I've had a customer phone because our
app would no longer run.  "What do you mean", I asked.
It turned out the shortcut icon on the desktop had
disappeared.  Well, I talked them through setting up
the shortcut again.  Can you believe it?  The shortcut
disappeared and _they_ didn't remove it (so they said, but
I suspect they did somehow by accident).  I know our
program didn't remove it.  Would Linux really cause me more
trouble than that?)

> Icon doesn't even have native ISAM file support, and that's
> just about a minimum pre-requisite for any serious business
> use (IMHO).

     Hmmm.  So it looks like I am back to the question of
whether all of the data would fit in RAM.  I think it would.
I'll try to run some tests.


> I believe that there are FoxPro (xBASE) flavor "server"
> engines available... which would satisfy this goal (FWIW)
> without requiring a complete application rewrite.

     This is a good point.  There are two products in
particular that I have heard good things about.  One is
Advantage Database Server.  It just costs money and requires
a dedicated NT (or Novell) server which costs money.  I
think we are talking about some thousands of dollars per
customer (that I think they do not wish to pay).  But,
Advantage gets rave reviews.  The other is RASQL/B which
provides an interface from Clipper to Btrieve.  Again,
substantial money for each customer.  Both of these seem
like overkill, remembering that our typical office has
two or three computers.

> >     2. Reduce network traffic (by not dragging the database
>
> There are other ways to do that, whether by thin-client
> approaches (ICK) or by things like ReachOut, PC-Anywhere, etc etc.

     Well, these solutions seem to require dedicating a
machine at a central office for use by remote office (and
possibly vice versa), since Windows isn't multi-user.  I
could see it for occasional use.  We do have one customer
using PC-Anywhere sometimes to access the office machine
from home.

> I think that, with 32Mb of RAM costing maybe $50 or
> thereabouts and 6Gb hard drives going for about $200-250,
> any discussion of saving bytes by increasing programmer
> effort is probably pretty silly.

     I generally agree with this.  My point was not so
much to save bytes on disk, but to allow the entire
database to reside in RAM.  I am not saying that is a
general solution for every business application, just
that it might well fit our application considering the
size of our databases.  Then, bang!, with it all in RAM,
certain complexities seem to disappear.  Well, I need
to do some testing.  (And, although 32MB of RAM might
seem cheap at $50, there is also the problem of how to
get it put into the machine.  Will the customer do that
himself?  I don't want to have to do it for him, but I
might have to.  So, really, the cost is greater than just
the $50.)

> If you're really worried about stuff like this, let me
> suggest that (instead of application-specific forms of
> compression like this) you'd be *far* better off to use
> Stacker or even Microsoft's own disk compression software.

     It will be a cold day in hell before I use Microsoft's
disk compression.  (Do I feel a chill in the room?).  Maybe
I'm living too far in the past, but disk compression has
caused too many problems.  Is it really reliable now?  I'm
still very suspicious of all the disk compression products.
Certainly (until convinced otherwise), I would buy larger
hard disks first and recommend the same to my customers.

> It's also a big PITA (doing it yourself) when it comes time
> to update records, since compressed records usually are
> nontrivial to update in place.

     Well, I've got to think all this through.  But, if
the data would indeed fit in RAM, wouldn't Icon take care
of the updating in place automatically?  I'd have the
responsibility of initially loading the data from disk
at the start of the day (or the start of the year or the
start of the hour depending on which operating system was
in use) and writing it back out later.  I envision writing
out log files as I go so the database could be reconstructed
in case of a power failure, etc.

> I hope this helps you!

     Yes.  At least it makes me think and reconsider my
opinions and prejudices.


> The right tool for the right job.

     It is hard to argue with that.  On the other hand,
it is not impossible.  There are costs to sticking entirely
with Clipper.  There are costs to change.  Which will be
best considering the short term and the long term?

> The right tool for the right job.

     It seems possible that there could be a language,
possibly Icon or Python, or possibly a combination of two
languages, that might generally be good enough for most
programming tasks even if it wasn't the absolute best
for any single one of those tasks, and not _too_ bad
for all the other tasks so that I would be better off
doing _everything_ in that one language (or two) when
you consider the efficiencies of working with just one
or two languages instead of 30 (fill in the blank) perfect
languages each perfect for its own task.  This is just
a conjecture, but it seems likely to me to be true.


  -- Frank  (in sunny and hot San Marcos, Texas)

  [e-mail]
  http://www.eskimo.com/~pygmy   (for the Stress Testing article)